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Abstract. We consider a basic problem in the general data streaming model, namely, to esti- 
mate a vector / £ Z n that is arbitrarily updated (i.e., incremented or decremented) coordinate- 
wise. The estimate / £ Z™ must satisfy ||/ — / ||oo < that is, Vi (\fi — fi\ < e||/||i)- It is 



known to have 0(e 1 ) randomized space upper bound [3], f?(e 1 log(en)) space lower bound 
^s^j . [2] and deterministic space upper bound of fl(e~ 2 ) bitsfj We show that any deterministic 



algorithm for this problem requires space fl(e (log||/||i)) bits 



1 Introduction 



5h ■ 

<: 
\o : 

A data stream a over the domain [1, n] ={1,2,..., n} is modeled as a sequence of records of 
U: the form (pos, i, Sv), where, pos is the current sequence index, i £ [1, n] and Sv £ {+1, — 1}- 

^-^ ' Here, Sv = 1 signifies an insertion of an instance of i and Sv = — 1 signifies a deletion 

of an instance of i. For each data item i £ [1,n], its frequency (freq a)i is defined as 
J2( P os,i,8v) e stream ^ The size of (T is defined as \a\ = max{||freq cr'Hoo | a' prefix of a}. In 
this paper, we consider the general stream model, where, the n-dimensional frequency vector 
J> ■ freq a £ IT 1 . The data stream model of processing permits online computations over the 

input sequence using sub-linear space. The data stream computation model has proved to 
be a viable model for a number of application areas, such as network monitoring, databases, 
t— i 1 financial data processing, etc.. 

We consider the problem ApPROxFREQ(e): given a data stream a, return /, such that 
err(f, freq a) < e, where, the function err is given by ([1]). Equivalently, the problem may 
be formulated as: given i £ [l,n], return fa such that |/j — (freq a)i\ < e ■ 1 1 freq a\\i, where, 

ll/l|l=Efe[l,n]l/*l- 

. ^ / ? j.\ def ||/ — /||oo , /,i 

The problem AppROxFREQ(e) is of fundamental interest in data streaming applications. 
For general streams, this problem is known to have a space lower bound of f2(e~ 1 log(ne)) 
j2], a randomized space upper bound of 0(e _1 ) [4J, and a deterministic space upper bound of 
0(e~ 2 ) bits [Tj- For insert-only streams (i.e., freq a > 0), there exist deterministic algorithms 
that use 0((e _1 )(log(mn))) space [5I11|12] : however extensions of these algorithms to handle 
deletions in the stream are not known. 

Mergeability. Data summary structures for summarizing data streams for frequency de- 
pendent computations (e.g., approximate frequent items, frequency moments, etc.; formally 
defined in Section [2]) typically exhibit the property of arbitrary mergeability. If D is a data 
structure for processing a stream and Dj, j = 1, . . . , k for k arbitrary, be the respective cur- 
rent state of the structure after processing streams Sj, then, there exists a simple operation 
Merge such that Merge(D\, . . . , Dk) reconstructs the state of D that would be obtained by 



* This is the full version of the paper with the same title in Proceedings of the Third International Computer 

Science Symposium in Russia (CSR-2008). 
1 The O and Q notations suppress poly-logarithmic factors in n,loge _1 , ||/||oo and log<5 _1 , where, 8 is the 

error probability (for randomized algorithm). 



processing the union of streams Sj, j = 1, 2, . . . , k. For randomized summaries, this might 
require initial random seeds to be shared. Thus, a summary of a distributed stream can 
be constructed from the summaries of the individual streams, followed by the Merge op- 
eration. Almost all known data streaming structures are arbitrarily mergeable, including, 
sketches pQ, Countsketoh [3], Count-Min sketches [I], Flajolet-Martin sketches [6] and 
its variants, fc-set[S], CR-precis structure [Zj and random subset sums [TO]. In this paper, 
we ask the question, namely, when are stream summaries mergeable? 

Contributions. We present a space lower bound of i?(e~ 2 (log m)) — O(logra) bits for any 
deterministic uniform algorithm A n for the problem AppROxFREQ(e) over input streams 
of size m over the domain [l,n], where, \/{2A^ l /n) < e < 1/32. The uniformity is in the 
sense that A n must be able to solve ApPROxFREQ(e) for all general input streams over the 
domain [l,n]. The lower bound implies that the CR-precis structure [7] is nearly space- 
optimal for ApPROxFREQ(e), up to poly-logarithmic factors. The uniformity requirement is 
essential since there exists an algorithm that solves AppROxFREQ(e) for all input streams 
o~ with \a\ < 1 using space 0(e -1 polylog(n)) [9]. 

We also show that for any deterministic and uniform algorithm A n over general streams, 
there exists another algorithm B n such that (a) the state of B n is arbitrarily mergeable, 
(b) B n uses at most O(logre) bits of extra space than A n , and, (c) for every input stream 
a, the output of B n on a is the same as the output of A n on some stream a 1 such that 
freq a = freq a'. In other words, if A n correctly solves a given frequency dependent problem, 
so does B n \ further, the state of B n is arbitrarily mergeable and B n uses O(logn) bits of 
extra space. This shows that deterministic data stream summaries for frequency dependent 
computation are essentially arbitrarily mergeable. 

2 Stream Automaton 

In this section, we define a stream automaton and study some basic properties. 

Definition 1 (Stream Automaton). A stream automaton A n over the domain [1, n] is 
a deterministic Turing machine that uses two tapes, namely, a two-way read-write work- 
tape and a one-way read-only input tape. The input tape contains the input stream a. After 
processing its input, the automaton writes an output, denoted by output 4 (c), on the work- 
tape. □ 

Effective space usage. We say that a stream automaton uses space s(n,m) bits if for all 
input streams a having |er| < m, the number of cells (bits) on the work-tape in use, after 
having processed a, is bounded by s(n,m). In particular, this implies that for m > m', 
s(n, m) > s(n, m'). The space function s(n, m) does not count the space required to actually 
write the answer on the work-tape, or to process the s(n, m) bits of the work-tape once the 
end of the input tape is observed. The proposed model of stream automata is non-uniform 
over the domain size n, (and uniform over the stream size parameter m = |cr|), since, for 
each n > 1, there is a stream automata A n for solving instances of a problem over domain 
size n. This creates a problem in quantifying effective space usage, particularly, for low- 
space computations, that is, s{n,m) = o(n log m). Let Q{A n ) denote the set of states in the 
finite control of the automaton A n . If |Q(A n )| > m2 n , then, for all m! < m, the automaton 
can map the frequency vector isomorphically into its finite control, and s(n,m) = 0. This 
problem is caused by non-uniformity of the model as a function of the domain size n, and 
can be avoided as follows. We define the effective space usage of A n as 

Space(A n ,m) = f s(n,m) + log s(n,m) + |Q(A n )| . 



Although, the model of stream automata does not explicitly allow queries, this can be 
modeled by a stream automaton's capability of writing vectors as answers, whose space is 
not counted towards the effective space usage. So if {qi}i^i denotes the family of all queries 
that are applicable for the given problem, where, I is a finite index set of size p(n) then, the 
output of the automaton can be thought of as the p(n)-dimensional vector output^ (<r). 

A frequency dependent problem over a data stream is characterized by a family of binary 
predicates P n (/,freq a), f G Z p( - n \ n > 1, called the characteristic predicate for the domain 
[l,n\. P n defines the acceptability (or good approximations) of the output. A stream au- 
tomaton A n solves a problem provided, for every stream a, P n (output^ n (<r),freq a) holds. 
For example, the characteristic predicate corresponding to the problem AppROxFREQ(e) is 
err(f, f) < e, where, / G Z n and err(-, •) is defined by ([1]). Examples of frequency dependent 
problems are approximating frequencies and finding frequent items, approximate quantiles, 
histograms, estimating frequency moments, etc.. 

Given stream automata A n and B n , B n is said to be an output restriction of A, provided, 
for every stream a, there exists a stream a' such that, freq a = freq a' and output Bn (a) = 
output^ (<t'). The motivation of this definition is the following straightforward lemma. 

Lemma 1. Let P n be the characteristic predicate of a frequency- dependent problem over 
data streams and suppose that a stream automaton A n solves P n . If B n is an output restric- 
tion of A n , then, B n also solves P n . □ 

Proof. Let a be any input stream to B and let / = output B (<r) be the output of B on a. 
Since, B is an output restriction of A, hence, / = output^ (a) , for some stream a. Since, A 
solves P, therefore, (/,freq a') G P. However, freq a' = freq a, and therefore, (/,freq a) G P. 
Since, this holds for all a, B solves P as well. □ 

Notation. Fix a value of the domain size n > 2. Each stream record of the form (i , 1) and 
(i, —1) is equivalently viewed as and and — respectively, where, = [0, . . . , 0, 1 (position 
i),Q... ,0] is the i th standard basis vector of W 2 . A stream is thus viewed as a sequence 
of elementary vectors (or its inverse). The notation a o r refers to the stream obtained 
by concatenating the stream t to the end of the stream a. In this notation, freq = e^, 
freq — = — and freq a o r = freq a + freq r. The inverse stream corresponding to 
a is denoted as o r and is defined inductively as follows: e[ = — e^, —e\ = and and 
(a o r) r = r r o a r . The configuration of A n is modeled as the triple (q, h, w), where, q is the 
current state of the finite control of A n , h is the index of the current cell of the work tape, 
and w is the current contents of the work-tape. The processing of each record by A n can be 
viewed as a transition function ®A n { a i v ), where, a is the current configuration of A n , and v 
is the next stream record, that is, one of the e^'s. The transition function is written in infix 
form as a (Ba„ v- We assume that ®A n associates from the left, that is, a ®A n ui U2 means 
(a©A n ui) ®A n U2- Given a stream automaton A n , the space of possible configurations of A n 
is denoted by C(A n ). Let C m (A n ) denote the subset of configurations that are reachable 
from the initial state o and after processing an input stream a with |<t| = 1 1 freq (t||oo < m - 
We now define two sub-classes of stream automata. 

Definition 2. A stream automaton A n is said to be path independent, if for each con- 
figuration s of A n and input stream a, s®A n o~ is dependent only on freq a and s. A stream 
automaton A n is said to be path reversible if for every stream a and configuration s, 
s ®A n °~ ° o~ r = s, where, a r is the inverse stream of a. □ 

Overview of Proof. The proof of the lower bound on the space complexity of AppROxFREQ(e) 
proceeds in three steps. A subclass of path independent stream automata, called free au- 



tomata is defined and is proved to be the class of path independent automata whose tran- 
sition function ®A n can be modeled as a linear mapping of M n , with input restricted to Z n . 
We then derive a space lower bound for AppROxFREQ(e) for free automata (Section 14. II) . In 
the second step, we show that a path independent automaton that solves AppROxFREQ(e) 
can be used to design a free automaton that solves AppROxFREQ(4e) (Section I4.2[) . In 
the third step, we prove that for any frequency-dependent problem with characteristic 
predicate P n and a stream automaton A n that solves it, there exists an output-restricted 
stream automaton B n that also solves P n , is path- independent, and, Space(-B n ,m) < 
Space(A n ,m) + O(logn). This step has two parts — the property is first proved for the 
class of path-reversible automata A n (Section [5j) and then generalized to all stream au- 
tomata (Section [6]). Combining the results of the three steps, we obtain the lower bound. 

3 Path-independent stream automata 

In this section, we study the properties of path independent automata. Let A n be a path- 
independent stream automaton over the domain [l,n] and let © abbreviate ®A n - Define 
the function + : Z n x C(A n ) -> C(A n ) as follows. 

x + a = a © a where, freq a = x . 

Since A n is a path independent automaton, the function x + a is well-defined. The kernel 
Ma„ of a path independent automaton is defined as follows. Let the initial configuration 
be denoted by o. 

M An = {x G Z n | x + o = + o} 

The subscript A n in Ma„ is dropped when A n is clear from the context. 
Lemma 2. The kernel of a path independent automaton is a sub-module ofZ n . 

Proof. Let x £ M. Then, + o = —x + x + o = —x + o, or — x G M. If x, y £ M, then, 
+ o = x + o = x + y + o, or, x + y £ M. So M is a sub-module of Z n . □ 

The quotient set Z n /M = {x + M \ x £ Z n } together with the well-defined addition 
operation (x + M) + (y + M) = {x + y) + M, forms a module over Z. 

Lemma 3. Let M be the kernel of a path independent automaton A n . The mapping x + 
M i— ► x + o is a set isomorphism between Z n /M and the set of reachable configurations 
{x + o | x G Z n }. The automaton A n gives the same output for each y G x + M , xSZ". 

Proof, y £ x + M iff x — y £ M or — y + x + o = o, or, x + o = y + o. Thus, A n attains the 
same configuration after processing both x and y and therefore A n gives the same output 
for both x and y. Since, x + o = y + o iS x — y £ M, which implies that the mapping 
x + Mi— >x + oisan isomorphism. □ 

Let denote the subset {— m, . . . , m} n of Z n . 

Lemma 4. Let A n be a path independent automaton with kernel M . Then, 

Space(A n ,m) > \ log|{x + M | x £ Z™ }| ] > (n - dimM) log(2m + 1). 



Proof. The set of distinct configurations of A n after it has processed a stream with frequency 
x G Z^ is isomorphic to {x + M \ x G Z^}. The number of configurations using workspace 
of s = s(n,m) is at most \Qa„ \ • s ■ 2 s . Therefore, 

2 Space(A n ,m) = |g^J . s . 2 * > | {x + M | X G Z™ } | . (2) 

We now obtain an upper bound on the size \M n Z^J. Let b\, b%, ■ ■ ■ , b r be a basis for M. 
The set 

Pm = {ai&i + • • • + OL r b T | \ai\ < m and integral, i = 1, 2, . . . , n} 

defines the set of all integral points generated by b%, bi, ■ ■ ■ , b r with multipliers in {— m, . . . , m}. 
Thus, 



It follows that 



MnZ^| < \P m \ = (2m + l) r . (3) 



\{x + M\xeZ- m }\>-L^>(2rn + ir 



Since, r = dim M , substituting in ([2]) and taking logarithms, we have 

Space(A n ,m) > log|{x + M | x G Z™}| > (n-r)log(2m + 1) . □ 

Lemma shows that given a sub- module M, a path- independent automaton with a given 
M as a kernel can be constructed using nearly optimal space. The transition function 
(x + M) + (y + M) = (x + y) + M implies that the state of a path independent automaton 
is arbitrarily mergeable. 

Lemma 5. For any sub-module M of7L n , one can construct a path-independent automaton 
with kernel M that uses nearly optimal space s(n, m) = log|{a; + M \ x G [— m . . . m] n }\ + 
O(logn) and uses states in its finite control. □ 

Proof. Let M be a given sub- module of Z n with basis b\ , . . . , b r (say) . It is sufficient to con- 
struct a path independent automaton whose configurations are isomorphic to E = 7* n /M. 
Since, Z n is free, Z n /M is finitely generated using any basis of Z n . Therefore, the basic 
module decomposition theorem states that 

Z n /M = Z/(gi) • • • © Z/(g r ) . (4) 

where, | <?2 1 • • " (Here, © refers to the direct sum of modules.) The finite control of 
the automaton stores qi , . . . , q r and the machinery required to calculate 1 mod qj and — 1 
mod qj for each j. For the frequency vector /, the residue vector / + M is maintained as a 
vector of residues with respect to the r^'s as given by dU). Since, ([!]) is a direct sum, hence, 
the space used by this representation is optimal and equal to \{x + M \ x G [— m . . . m] n }|. 

□ 

Definition 3 (Free Automaton). A path independent automaton A n with kernel M is 
said to be free ifZ n /M is a free module. □ 

That is, A n is free if for every x G Z n such that there exists a G Z, a ^ and ax G M, it 
is the case that x G M. For free automata A n , it follows that Z n is the direct sum of M 
and Z n /M, that is, Z n = Z n /M M. For the ApproxFreq problem and other related 
problems, it will suffice to consider only free automata!]. Lemma[6]shows that the transition 
function © of a free automata can be represented as a linear mapping. 



2 There exist stream automata that use finite field arithmetic and consequently have torsion, for example 
0. 



Lemma 6. Let A n be free automaton with kernel M . There exists a unique vector subspace 
M e ofW 1 of the smallest dimension containing M. The mapping x + M ^ x + M e is an 
injective mapping from Z n /M to R n /M e . 7/dimZ n /M = r, then, there exists an orthonor- 
mal basis V = [Vi, V2] of R n such that rank(Vi) = r, rank(V2) = n — r, M e is the linear 
span 0/V2 and R n /M e is the linear span ofV\. □ 

Proof. Z is a principal and entire ring. Since Z n is a module over Z, its sub-modules are 
free modules. Therefore, M is a free module. Since Z n /M is given to be free, Z n is the 
direct sum of two free modules, Z n = Z n /M M. Therefore, both M and Z n /M have 
bases, say B\ and B2 whose union is a basis for Z n . Since, Z ra is a free module and has the 
standard n-dimensional basis e±, . . . , e n , therefore, all bases of Z n have the same dimension. 
Without loss of generality, therefore, let B = [b\, 62, . . . , b n \ be a basis of Z n such that 
B-z = [bi, . . . , b r \ is a basis for M and B\ = [b r+ i, . . . , b n ] is a basis for Z ra /M. 

Let M e denote the span of bi, . . . , b r over R. M e is obviously the smallest vector space 
over R that contains M, since, every vector space over R containing M must contain 
the span of b±, . . . , b r . Therefore, dimM e < r and therefore, dimR n /M e < n — r (same 
argument). However, the standard basis {e\, . . . , e n } is a basis of Z n and therefore, dimM e + 
dimR n /M e = n. Hence, dimM e = r and dimR n /M 6 = n — r. Further, b±, . . . , b n continues 
to be a basis for R n , of which b±, . . . ,b r is a basis for M e and b r+ ±, ■ ■ ■ ,b n is a basis for 
R n /M e . 

Consider the mapping x + M ^ x + M e . Let x, y denote the elements x + M and y + M 
of Z n /M. Suppose that x 7^ y. Then, x — y^ M. x — y can be expressed uniquely as a linear 
combination of the basis elements. 

n 

x-y = ^Qj6j, a.i £ Z 
i=i 

Hence, x—y has the same unique representation in the vector space over R n . Further, at least 
one of the coordinates a±, . . . ,a r is non-zero, otherwise, x — y would belong to M. Since, 
x — y has the same representation in the vector space R n , x — y is not in M e . The mapping 
x + M x + M e is therefore injective. Using standard Gram-Schmidt orthonormalization 
of B\ and B2 respectively viewed as defining vector sub-spaces over R, we get V± and Vi- 
By the previous argument, rank(Vi) = n — r and rank(V^) = r. □ 



4 Frequency estimation 

In this section, we present a space lower bound for AppROxFREQ(e) using path-independent 
automaton. Recall that a stream automaton A n solves AppROxFREQ(e), provided, after 
processing any input stream a with freq a = x, A n returns a vector x G R ra satisfying 
err(x, x) = < e - I n general, if an estimation algorithm returns the same estimate u 

for all elements of a set S, then, err(u, S) is defined as max^gs err(u, y). Given a set S, let 
min^S) denote the element in S with the smallest i\ norm: mm£ 1 (S) = argmin^g^ \\y\\i- 

Lemma 7. If S C Z n and there exists h G R™ suc/i £/iat err(/i, S 1 ) < e, then err(min^ 1 (S) , 5) < 
2e. 

Proof. Let 5 denote min^S) and y £ S. Since, < ||y||i, by triangle inequality, 

err(<7, y) = — — < — — 1 — < - — 1 ^""n < e + e = 2e □ 

Wvh \\y 1 b 1 ji 11 



4.1 Frequency estimation using free automata 

In this section, let A n be a free automaton with kernel M that solves the problem AppROxFREQ(e). 



Lemma 8. Let M be a sub-module of TU 1 . (1) if there exists h such that err(h,M) < e, 
then, err(0,M) < e, and, (2) if err(0,M) < e then err(0,M e ) < e. 

Proof (of Lemma \3part (!))■ For any y^ £ Z, max(|/ij — Ui\, \hi + yi\) > \y%\- Therefore, 

max(||/i - y||oo, ||/i + y||oo) > Hylloo . 
Let y £ M. Since, M is a module, — y £ M. Thus, 

err(0, y) = err(0, -y) = < tt\- max(||ft - y||oo, \\h + y\\oo) 

\\y\h \\y\h 

= max(err(h, y), err(h, —y)) < e □ 

Proof (of Lemma part (2)). Let z £ M e . Let bi,b 2 , ■ ■ ■ ,b r be a basis of the free module 
M. For t > 0, let tz be expressed uniquely as tz = ot\b\ + . . . + a r b r , where, a^s belong 
to R. Consider the vertices of the parallelopiped Pt z whose sides are b\, b 2 , ■ ■ ■ , b r and that 
encloses tz. 



Ptz = [ai]h + [a 2 ]b 2 + . . . + [a n ]b n 

+ {Pih + f3 2 b 2 + ■ ■ ■ + Prb r | P 3 e {0,1}, j = 1,2,. ..,r} 

where, [a] denotes the largest integer smaller than or equal to a. Since, £oq is a convex 
function ||£z||oo < \\y\\ao for some y £ P tz . Let y = Y7j=i Pj b ji f° r Pj e {°> 1}> J = 1, 2, . . . , r. 

r r r 

j=l j=l j=l 



or, 



\tz\\i > ||y||i - y^||&jl|i 



Therefore, 



err(0, tz) 




where, the last step follows from the assumption that y £ M and therefore, err(0, y) = 
^|^° < e. The ratio ^j^^ 1 can ^e made arbitrarily small by choosing t to be arbitrarily 

large. Thus, lim^oo err(0, tz) < e. Since, err(0,tz) = ^Jj | ^° = ^J j| ^ = err(0,z), for all L, we 
have, err(0, z) < e. □ 

Lemma 9. Let A n be a free automaton that solves AppROxFREQ(e) and has kernel M . 
Let M e be the smallest dimension subspace o/R n containing M. Let V\, V 2 be a collection 
of vectors that forms an orthonormal basis for R n such that V 2 spans M e and V\ spans 
R n /M e . Then, for 1/V6n~ < e < \, rank(Vi) > j^- 



Proof. Since, V\ has orthogonal columns 

II W T e*ll2 = \\V?ei\\l = {VxVfe^ . (5) 

Therefore, 

n n 

traced Vf) = YjYxV^e^i = X^I^^Hi 

i=l i=l 

The trace of ViV^ is the sum of the eigenvalues of ViV^ . Since, V\ is orthogonal columns 
and has rank rank(Vi), V\V^ has eigenvalue 1 with multiplicity rank(Vi) and eigenvalue 
with multiplicity n — rank(Vi). Thus, trace(ViV ] T ) = rank(Vi) = r (say). It follows that 

n 

r = traced Vf) = J^H W T e<||l . (6) 
i=l 

Further, 

n n 

y~]\\ViVi' ej\\i < y^||ViVi T ei||2\/n, since, ||rc||i < ||x||2\/n 
i=l i=i 

/ n \ V2 

— y^ll^i^ T ' e »ll2 n 1 / 2 , by Cauchy-Schwartz inequality 

iVfc by © . (7) 



Let 

J = {ViVfei | 1 < i < n and ||T^.V^ r e*[|| < 3r/n}, and 
if = {ViV^f* | 1 < i < n and HVitfeiHi < 3v^} . 

Therefore, by © and {7|, 

2n , , 2n 
\J\ > y and \K\ > y ■ 

Hence, JnK^cj), that is, there exists i such that || Vi V^e^ H2 < (3r /n) 1 / 2 and || ViV^eiHi < 
3-y/r. Since, e, - ViV^e* = V 2 V<fei £ M e , therefore, 

e^enfo-Wf *,())- H e * " 



Therefore, 



By ©, 



Therefore, 



|e* - ^ilf eiHoo < e^Vfe* - e^i . (8) 



n 



\ei - VxVfeiWn > {(ei-ViVfeiU = Hl^tfe^ > 1- — , by © and since V^e, e J . 

n 



Substituting in ((5J), 



1 < \\ei — V\Vi eiWvz < e\\ViV^ei — ej||i < e(|| V^V^ei || + l), by triangle inequality 

< e(3Vr + 1), since, ViV^e* G K . 

Simplifying, r > min (n/6, l/(36e 2 ) - l/9e)). Therefore, for 1/y/Gn < e < ±, r > □ 
Lemma 10. Let ttK= < e < |. Suppose A n be a free automaton that uses s(n,m) bits on 



the work-tape to solve AppROxFREQ(e). Then, s(n,m) = Q 




Proof. Let M = kernel of A n . By Lemma[9l rank(Vi) = n — dimM e = Q (^)- By LemmaUl 
s(n,m) = l?((n — dim M) log m) . Since, dimM = dimM e , the result follows. □ 

4.2 General path independent automata 

We now show that for the problem AppROxFREQ(e), it is sufficient to consider free au- 
tomata. Let A n be a path-independent automaton that solves AppROxFREQ(e) and has 
kernel M. Suppose that Z n /M is not free. Let M' be the module that removes the torsion 
from Z n /M, that is, 

M' = {x G Z n | 3a G Z, a ^ and ax G M} . (9) 
Lemma 11. Z n /M' is torsion-free. 

Proof (Of Lemma [771 ). Suppose y = y + M' is a torsion element in Z n /M'. Then, there 
exists 6 G Z and 6^0 such that 6y = by + M' G M' or that 6y £ -M 7 - Therefore, there exists 
a G Z, a ^ 0, such that by = ax, for some x G M, or that, y = (b~ 1 a)x with 6 _1 a ^ 0. 
Therefore, y G M. Hence, Z n /M' is torsion-free. □ 

Fact 12 Lei &i, b2, ■ ■ ■ ,b r be a basis of M' . Then, 3 cti, . . . , a r G Z— {0} smc/i that a\bi, . . . , a r 6 r 
is a basis for M. Hence, M e = {M') e . 

Proof ( Of Fact It follows from standard algebra that the basis of M is of the form 
a\b\, . . . , a r b r . It remains to be shown that the a^s are non-zero. Suppose that a± = 0. For 
any a G Z, a ^ 0, suppose ax G M and x G M' . Then, x has a unique representation as 
x = Yl T j=i x jbj- Thus, ax = X^=i( ax j)^j ^ an d has the same representation in the basis 
{ctjbj}j=i^,^ n . Therefore, ax\ = or x\ = for all x G M', which is a contradiction. 

Let {b±, &2> • • • , frr} be a basis for M' . Then, by the above paragraph, there exist non-zero 
elements a±, . . . , a r such that {a\b\, a2&2> • • • > ct r b r } is a basis for M. Therefore, over reals, 
6 r ) = { ai bi,...,a r b r ). Thus, M e = (M') e . □ 

We show that if a path independent automaton with kernel M can solve AppROxFREQ(e), 
then a free automaton with kernel M' D M can solve AppROxFREQ(4e). 

Lemma 13. Suppose A n is a path independent automaton for solving AppROxFREQ(e) and 
has kernel M . Then, there exists a free automaton B n with kernel M' such that M' D M , 
Z n /M' is free, and err( min^ (x + M'),x) < 4e . 



Proof (Of Lemma [T3\). Let M be the kernel of A n and let M' be as denned in Q, so 
that Z n /M' is free. For x G IT 1 , define h(x + M') = mm ei (x + M'). Let y G x + M' . 
Then, y G x\ + M for some a?i. Let y = output^ {x\ + M) denote the output of A n for an 
input stream with frequency in x\ + M (they all return the same value, since, A n is path 
independent and has kernel M) and let y' = min^xi + M). Let h denote h(x + M') and 
let h = output^ (h + M). Therefore, 

err[n,y) = — — < — — 1 — 1 — (10) 

Wvh IMIi \\y\h Wvh 

The first and the second terms above are bounded by e as follows. The first term ^ N = 
err(y, y) < e, since, y G x\ + M and y is the estimate returned by A n for this coset. The 
second term 

\\y ~ y \\oo , 1 1 2/ 2/ lloo /a t\ , 
< — n — 77i — = err( - y ' y > - e 



\\y\\i \\y Hi 

since, ||y'||i < \\y\\i and y' lies in the coset x\ + M. The third term in (j 1 U j) can be rewritten 
as follows. By Lemma [TTl y' — h G M' and M' C M e . Therefore, 

\W - h\\oo . \\y' - hWoo \\y'-h\\i . ... 

— ij-n — < -jn — 7-n n — 771 — ' smce ' \\y 1 - Wvh 

\\y\h \\y-h\\i \\y'\\i 

< e ■ — ^7 - — — by Lemma [8] and by triangle inequality 

Wvh 

< 2e, since, \\h\\i < Wy'Wi 

By (|10p . err(h,y) < e + e + 2e = 4e. The automaton l? n with kernel M' is constructed as 
in Lemma [SJ □ 

Lemma 14. Suppose 2 ^/^ — 6 < 32 ■ ^ n ^ e a P a ^ in-dependent automaton that solves 
ApproxFreq (e). If A n has kernel M, then, n - dimM = 17 (4) • 

Proof. By Lemma [TUI there exists a free automaton A' n with kernel M' D M that solves 
AppROxFREQ(4e). Therefore, n - dimM > n — dimM' = fi (4r), by Lemma [TUl □ 



5 Path reversible automata 



In this section, we show that given a path reversible automaton A n , one can construct a 
path independent automaton B n that is an output restriction of A n and Space(S n ,m) < 
Space(A n ,m) + O(logn). Let A n be a path reversible automaton. For / G Z n , define 
^Anif) = {s \ 3a s.t. o © a = s and freq a = /}. The kernel of A n is defined as fol- 
lows: M = Ma„ = {/ I o G </>A n (/)}- Let C = C(A n ) be the set of reachable configurations 
from the initial state o of A n and let C m = C m (A n ) denote the subset of C(A n ) that are 
reachable from the initial state o on input streams a with |o~| < m. Define a binary relation 
over C as follows: s ~ t if there exists / G Z n such that s,t S 4>A n {f)- 

Lemma 15. i. M is a sub-module of "L n . 

2. If f - g E M then <f>A n (f) = 4>A n (g), and, if <j>A n (f) D 0A n (sO «s non-empty, then, 
f-geM. 

3. The relation ~ over C is an equivalence relation. 

4- The map [s] 1— ► / + M, /or s G 4>A n {f), is well-defined, 1-1 and onto. 



Proof ( Of Lemma 1 151 part 1.). Since the empty stream has frequency 0, G M. Suppose 
/ G M. There exists a such that freq cr = / and o © a = o. By path reversibility, o = 
o © a o a r = offi a r . Since freq a r = —freq a = —f, therefore, — / G M. Now suppose 
f,g£ M. Then there exists cr, r such that freq a = /, freq t = g, o® a = o and o © r = o. 
Therefore, o©<tot = oot = o. Since, freq o" o r = freq a + freq r = / + g, therefore, 
/ + <?GM. □ 

Proof ( Of Lemma\15\ part 2.). Suppose f = g + h, for some h G M. Then, there exists cr 
such that o©a = o and freq a = h. Let a G (pAnig) and let r be a stream such that offir = a 
and freq t = g. Then, o®a®T = o®T = a, and freq a © r = freq cr + freq t = h + g = f. 
Therefore, a G 4>A n (f), or, (pA n (g) C 4>A n {f)- Reversing the roles of / and 5, we have, 
4>A n (f) C 4>A n {g)i or that, (pA„(f) = 4>A n (g)- This proves the first assertion of the lemma. 
Conversely, Suppose a G <f>A n (f) H (f>A n {g)- Then, there exist streams cr and r such that 
freq a = f, freq t = g and o©cr = offir = a. By path reversibility, a © r r = o. Therefore, 
o © cr o r r = a o r r = 0, and freq (jor r = freq cr + freq r T = f — g. Therefore, o G (j>A n (/ — 5) 
and so / - g G M. □ 

Proof ( Of Lemma\15\ part 3.). By definition, ~ is reflexive and symmetric. Suppose that 
s ~ t and t ~ n. Then, there exists /,j £ Z" such that s,f 6 4>A n {f) and t,u £ 4>A n {g)- 
Therefore, i G 4> An (f )r\(j)A n (g)- Hence, f-g £ M and so 0a„(/) = ^(sO- Thus, s ~ n. □ 

Proof ( Of Lemma\T5\ part 4-)- Suppose s G (/)n</>A„(g), then, f—g G M, by LemmaPPol 
part 2, or that, / + M = g + M. Hence, the map is well-defined. Suppose [s] and [t] both 
map to / + M. Then, s,t G 4>A n (f), and so s ~ t and therefore, [s] = [i]. Hence the map is 
1-1. For / G Z n , (pA n (f) is non-empty and for any s G 4>A n (f), [s] maps to / + M, proving 
ontoness. □ 

Let B n be a path independent stream automaton whose configurations are the set of cosets 
of M and whose transition is defined as by the sum of the cosets, that is, / + (x + M) = 
(/ + x) + M, constructed using Lemma Its output on an input stream a is defined as: 

output B n (cr) = choice {output of A n in configuration s \ s G </>A n (freq a)} 

where, choice S returns some element from its argument set S. 

Lemma 16. B n is an output restriction of A n . 

Proof, f + M = g + M if and only if 4>A n (f) = 4>A n (<?)• Therefore, oute(cr) is well-defined. 
Further, by definition of out#, out^ (cr) = the output of A in some configuration s, where, 
s G 0A„(freq a). Thus, B n is an output restriction of A n . □ 

We can now prove the main lemma of the section. 

Lemma 17. Let A n be a path reversible automaton with kernel M . Then, there exists a 
path independent automaton B n with kernel M that is an output restriction of A n such that 
\og\C m (A n )\ +0(logn) > Space(B n ,m), form> 1. 

Proof. Let B n be constructed in the manner described above. By Lemma [TBI is an output- 
restriction of A n . Since the map [s] —* f + M, for s G </>A n {f) is 1-1 and onto (Lemma [T5l 
part 4), therefore, for every m, each reachable configuration of B n after processing streams 
cr with freq a G [— m . . . m] n can be associated with a disjoint aggregate of configurations of 
A n . The number of reachable configurations of B n after processing streams with frequency 
in [— m . . . m] n is \{x+M \ x G [— m . . . m] n }\. Thus, |C(A n )| > \{x+M \ x G [— m . . . m] n }\. 
By Lemma[5j Space(B n ,m) = log |{x + M \ x G [— m . . .m] n }\ + O(logn). Combining, we 
obtain the statement of the lemma. □ 



Remarks. The above procedure transforms a path reversible automaton A n to a path- 
independent automaton B n such that log|C m (^4 n )|+0(log n) > Space(-B„, m), for all m > 1. 
However, the arguments only use the property that the transition function ffi^ n is path 
reversible, and the fact that the subset of reachable configurations C m (A n ) on streams of 
size at most m is finite. The argument is more general and also applies to computation 
performed by an infinite-state deterministic automaton in the classical sense that returns 
an output after it sees the end of its input, with set of states C, initial state o and a path- 
reversible transition function ffi^ n . The above argument shows that such an automaton 
A n can be simulated by a path-independent stream automaton B n with finite control and 
additional space overhead of O(logn) bits, such that B n is an output-restriction of A n . We 
will use this observation in the next section. 

6 Path non-reversible automata 

In this section, we show that corresponding to every general stream automaton A n , there 
exists a path reversible automaton A' n that is an output-restriction of A' n , such that 
Space(^4 n ,m) > log|C m (^J|. By Lemma [T7\ corresponding to any path reversible au- 
tomaton A' n , there exists an output-restricted and path independent automaton B n , such 
that log|C m (A^)| > Space(-B n , m) — O(logn). Together, this proves a basic property of 
stream automata, namely, that, for every stream automaton A n , there exists a path- 
independent stream automaton B n that is an output-restriction of A n and Space(-B n , m) < 
Space(^4 n , m) + 0(log n). We construct the path-reversible automaton A' n only to the extent 
of designing a path-reversible transition function ®A' n i a set of configurations C(A' n ) and 
specifying the output of A' n if the end of the stream is met while at any s £ C(A' n ). As per 
the remarks at the end of the previous section, this is sufficient to enable the construction 
of the path- independent automaton B n from A' n . 

6.1 Defining reversible transition function from stream automata 

In this section, we present detailed (existential) construction of constructing a reversible 
transition function ©' = ®A' n from a given general stream automaton A n with transition 
function © = ©a„- Let C = C(A n ) denote the space of configurations of A n and let 
Cm = C m (A n ) denote the subset of C(A n ) that are reachable from o on input streams of 
size at most m. 

Consider a directed graph G = (C, E) where, C = C(A n ) is the set of vertices and there 
is a directed edge from s to i provided there is some stream a such that freq a = and 
s © a = t. Define the equivalence relation s ~c t if there is a directed path from s to t in G 
and vice-versa. Let [s]^ G denote the equivalence class to which a configuration s belongs. 
Define the equivalence class restricted to the vertices of C m as [s]^ Gm = [s]~ G H C m . An 
equivalence class [s]^ G that satisfies the property that for every stream a with freq a = 
and s © a G C m , we have s © a € [s]~ Gm , are called terminal equivalence classes. 

Lemma 18. For every m > 1 and u E C m , there exists s = s(u) reachable from u in G m 
such that [s]~ Gm is a terminal equivalence class. 

Proof ( Of Lemma \18[). Let uq be a vertex reachable from u in G m . If [iio]~ Gm satisfies 
the property stated in the lemma, then, we are done. Otherwise, there exists a such that 
freq a = and u\ = no © <r € C m — [uq\. We now iteratively construct the sequence 
[ u i]~ Gm ' I n 2]~ Gm , • • • , in this manner. Suppose that two equivalence classes in this sequence 



are the same, that is, suppose [wi]~ Gm = [%']~ G • Then, there exists a directed path from 
Uj to Uj and vice-versa and therefore, [«j]~ G = . . . = [uj]~ G , that is, the iteration termi- 
nates. Since, C m is finite, the iterated sequence of equivalence classes of ~c m terminates. 
The last equivalence class of this sequence satisfies the property of the lemma. □ 

Define the mapping a m : C m — ► C m as follows: a m {s) = some member of some terminal 
equivalence class reachable from s (for e.g., the member with least lexicographic value 
among all candidates). Fix s £ C and consider the sequence {a m (s)} m >i. If this sequence 
is finite, then, one can define a(s) to be a final element of the sequence. Otherwise, we use a 
standard technique of passing to the infinite case by associating s with 'consistent' infinite 
sequences s = {a m (s)} m >i- 

Lemma 19. For s E C, a(s) ©' ej o — a = a(s) and a(s) ©' — a o a = a(s). 

Proof (Of Lemma [Wi ). A configuration s is first identified with the infinite sequence, 
s = {o! m (s)} m >i. Recall that the definition of a m (s) allows flexibility in the choice of a 
terminal class of ~G m - We now ensure that the choices are made in a consistent manner 
as follows. For each m, there is a path P m (s) from s to a vertex in the equivalence class 
a m (s). By consistent choices across m, we mean that the P m +j(s) is an extension of the 
path P m (s), for each j > 0, and for each s G C. From now 

The transition function ©' is defined in two steps. First, we define an intermediate 
function ©i. 

s ffii ei = {a m (a m {s) © ej)} m >i (11) 

Sequences are allowed to have the undefined element _L, since, it is possible that s C m and 
hence a m (s) is not defined. However, if a m (s) is defined, then, a m+ j(s) is defined, for all 
j > 0. This implies that the undefined elements, if they occur, form a prefix of the sequence 
s. 

We now attempt to prove Lemma 1191 for the transition function ©i. Let mo be the 
smallest m for which a m (s) © is well-defined. Then, for all m > mo, both a m (s) © and 
a(a m (s)©ej)©— ej are also well-defined. The arguments in the finite case of Lemma fT9l hold 
for each member m > rriQ. The same can be said for a m (s) © — ej. Thus, the two sequences 

{a m (s)} m >i and {a m (a m (a m (s) © e*) © -ej)} m >i 

differ at most in a finite prefix, where, the RHS sequence may have more _L elements than 
the sequence on the LHS. 

To resolve this problem, we define a relation = between pairs of infinite sequences. 

{um}m>i — {vm}m>i if u m and v m differ in a finite initial prefix. 

A finite sequence u±, . . . ,u r is modeled as an infinite sequence u±, . . . , u r , u r , u r , . . . whose 
last term is repeated. It is straightforward to see that = is an equivalence relation on the 
family of sequences. It now follows that 

{a m (s)} m >i = {a m (a m (a m (s) © e$) © -ej)} m >i . 

For each configuration s in the original automaton, we associate it with [s]^ as follows. 



= [ {a m (s)} m >i ]~ 



The transition function ©' is now denned as follows. 



[s]^ © ei = [ {a(a m (s) © e;)} TO >i]^ and 
[s]=* © -ei = [ {a(a m (s) © -e;)} m >i]~ 

It now follows, by repeating the arguments in the previous paragraph, that 

[s]a* ©' ej o -e, = [s]=* . 

This proves Lemma [191 with a(s) defined as [ {a m (s)}m>i]^- □ 

The map s *— > a(s) maps s to a congruence class over the space of consistent infinite 
sequences. Define C' m = {(3(s) \ s E C m }. Therefore, \C' m \ < \C m \ for all m > 1. 

A path reversible automaton A' n is defined as follows. Initially A' n is in the state a{6). 
After reading a stream record (one of the e^'s or — e^'s), A' n uses the transition function ©' 
instead of © to process its input. However, s ©' a = a(s © a), where, a{t) is a set (possibly 
infinite) of states that cause A n to transit from configuration t on some input a' , with 
freq a' = 0. Equivalently, this can be interpreted as if a' has been inserted into the input tape 
just after A n reaches the configuration s and before it processes the next symbol-hence, A' n 
is an output-restriction of A n and is equally correct for frequency- dependent computations. 
This is the main idea of this construction. Thus, transitions of ©' are equivalent to inserting 
some specifically chosen strings o~i,CT2, ■ ■ each having freq = 0, after reading each letter 
(i.e., ±ej) of the input. The output of A' n on input stream a is identical to the output of 
A n on the stream a' , where, a 1 is obtained by inserting zero frequency sub-streams into it. 
Therefore, freq (a') = freq (a) and A' n is an output restriction of A n . By Lemma [191 the 
transition function ©' is path reversible. Let C = C(A' n ) and C' m = C m (A' n ). Since, a(s) 
is an equivalence class over C(A n ), the map s t— > a(s) implies that \C' m \ = \{a(s) \ s £ 
C m }\ < \C m \. Starting from A' n , one can construct a path independent automaton B n as per 
the discussion in Section [5j The arguments in this section do not show that the transition 
function ©' can indeed by realized by a Turing machine that has only finite control. This is 
sufficient however, since, the path reversibility of ©' is only used to allow the techniques of 
Section[5]to be applicable, and hence to be able to construct a coset-based path independent 
automaton. Since any coset based automaton can be realized using finite number of states 
in its finite control (Lemma HI therefore, the final path-independent transition function is 
actually a stream automaton B n .) Theorem [T] summarizes this discussion. 

Theorem 1 (Basic property of computations using stream automata). For every 
stream automaton A n , there exists a path-independent stream automaton B n that is an 
output-restriction of A n and Space(B n ,m) < Space(A n ,m) ©O(logn). 

Proof. Let ©' be the transition function of the path-reversible automaton constructed as 
described above and let B n be the path-independent automaton obtained by translating ©' 
using the procedure of Section [5j Let C m and C' m denote the number of reachable configu- 
rations of A n and A' n , respectively, over streams with frequency vector in [— m . . . m] n . Let 
sa = SA(n,m). Let M be the kernel of B n . Then, 

\Qa\s A 2 sa > \C m \ > \C' m \ >\{x + M\xe [-m... m] n }\ > (2m + l)-^M 

where, the last two inequalities follow from Lemma[T71 Taking logarithms, Space(A n , m) > 
log|{x + M | x E [— m . . . m] n }\ > Sp&ce(B n ,m) — O(logn), by LemmaEl □ 



Theorem 2 (Lower bound for AppROxFREQ(e)). Suppose that < e < ^ and let 
A n be a stream automaton that solves AppROxFREQ(e). Then, Space(A n ,m) = 17 ^ lo g^' m ^ _ 
O(logn). 

Proof. By Theorem [TJ there exists a path independent automaton B n that is an output- 
restriction of A n and Space(j4 n ,m) > Space(i3 n ,m) — O(logn). By Lemma [TJ B n solves 
AppROxFREQ(e). If M is the kernel of B n , then by Lemma EE Space(£> n ) = l?((n - 
dimM)(log(2m + 1)). By LemmaQH n - dimM = Q (e~ 2 )- Thus, 

Space(A n ,m) = /2((n - dimM) logm) - O(logn) = Q -O(logn) . □ 

Since, any path-independent automaton is arbitrarily mergeable (see text before Lemma|5|, 
Theorem [1] implies that for any stream automaton A n , there exists an output-restricted 
automaton B n such that Space(5 n ,m) < Space(^4„, m) + O(logra), and the state of B is 
arbitrarily mergeable, establishing the claim made in Section [TJ 
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